Methods and systems for a natural and realistic telepresence experience

ABSTRACT

Embodiments of the present invention are directed towards methods and systems for providing an enhanced telepresence experience to users participating in a videoconferencing session (VCS). In the embodiments a camera is configured and arranged to capture image data covering objects within a substantial portion of the field of view (FOV) of a display device, without capturing image data encoding images displayed on the display device. That is, the camera&#39;s FOV is aligned with the display device&#39;s FOV. As such, the camera captures image data encoding images in a substantial portion of the display&#39;s FOV. According, users within a VCS may approach their display without falling outside their camera&#39;s FOV. This provides an enhanced telepresence experience, where the users may interact with each other through what appears to be a transparent window or barrier.

CROSS REFERENCE TO RELATED APPLICATIONS

The application claims priority to U.S. Nonprovisional application Ser.No. 16/046,628 filed Jul. 26, 2018, now U.S. Pat. No. 10,701,308,entitled, “METHODS AND SYSTEMS FOR A Natural and REALISTIC TELEPRESENCEEXPERIENCE,” which claims priority to U.S. Provisional PatentApplication No. 62/539,350, filed on Jul. 31, 2017, entitled METHODS ANDSYSTEMS FOR A NATURAL AND REALISTIC TELEPRESENCE EXPERIENCE, which isincorporated herein by reference in its entirety.

BACKGROUND

The proliferation of networked computing devices equipped with camerashas enabled users to frequently participate in real-time video-enabledteleconferencing sessions, such as conventional videoconferencingsessions (VCSs). Some devices (e.g., conventional mobile devices,desktops, and laptops) include a front-facing camera (FFC) (i.e., a“selfie” camera) that is configured to, as a user views a display of thedevice, capture image data of the user. Other conventional computingdevices may employ a standalone auxiliary camera, such as a web-enabledcamera (i.e., a webcam), that is communicatively coupled to thecomputing device and faces the user. Such conventional FFCs and webcamsenable users participating in a VCS to view one another, while remotelycommunicating in real-time. For example, users may each employ a largecamera-equipped display screen to remotely conduct a business meeting.The display employed in such a conventional arrangement may be aninteractive screen enabled to simultaneously display shared multi-mediacontent and video of the participants. The screen may take up asignificant portion of an office wall and be sized to provide video thatat least approximates the real-world physical dimensions of the usersand objects in their environment.

Such conventional VCSs have proven very useful in facilitating businessmeetings across distances, as well as personal communications betweenpeoples. However, such conventional VCSs are limited in providing therealistic experience of face-to-face communications between the users.That is, a VCS employing conventional cameras and displays may fail toprovide the users with a natural telepresence experience that at leastsimulates a realistic face-to-face interaction between the users.

For example, although both the camera and display may be facing in thesame direction, regions near the display may be outside of the camera'sfield of view (FOV). As such, when a user is outside their camera's FOV(e.g. the user is too close to the display), the camera will not captureimage data depicting the user. This is often experienced when a useremploys their hand and/or fingers to point to content rendered on thedisplay, walks too close to the display, and the like. For these andother reasons, conventional VCSs often lack a realistic and naturaltelepresence experience for the users.

SUMMARY

Embodiments of the present invention are directed towards methods andsystems for providing an enhanced telepresence experience to usersparticipating in a videoconferencing session (VCS). In the embodiments acamera is configured and arranged to capture image data covering objectswithin a substantial portion of the field of view (FOV) of a displaydevice, without capturing image data encoding images displayed on thedisplay device. That is, the camera's FOV is aligned with the displaydevice's FOV. As such, the camera captures image data encoding images ina substantial portion of the display's FOV. According, users within aVCS may approach their display without falling outside their camera'sFOV. This provides an enhanced telepresence experience, where the usersmay interact with each other through what appears to be a transparentwindow or barrier.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A schematically illustrates a conventional videoconference systemthat a user may employ to participate in a conventional videoconferencing session.

FIG. 1B schematically illustrates an enhanced videoconference systemthat a user may employ to participate in an enhanced video conferencingsession that is consistent with the various embodiments herein.

FIG. 1C illustrates an exemplary embodiment of an enhancedvideoconference system that is enabled to provide users with a realisticand natural telepresence experience that is consistent with the variousembodiments discussed herein.

FIG. 2 schematically illustrates an exemplary embodiment of an enhancedvideoconferencing apparatus that may be employed in the system of FIGS.1B-1C.

FIG. 3A schematically illustrates a video conferencing session betweenusers that is enabled via an enhanced videoconference system that isconsistent with the various embodiments herein.

FIG. 3B schematically illustrates another embodiment of an enhancedvideoconferencing apparatus that may be employed in the system of FIG.3A.

FIG. 4 illustrates one embodiment of a process flow for providingenhanced telepresence experiences that are consistent with the variousembodiments presented herein.

FIG. 5A shows a schematic view of projection-based enhancedvideoconferencing apparatus that is consistent with the variousembodiments.

FIG. 5B schematically illustrates another embodiment of an enhancedvideoconferencing apparatus that is consistent with the variousembodiments herein.

FIG. 6A schematically illustrates an enhanced integrated camera/displaydevice that may be employed in the various embodiments.

FIG. 6B schematically illustrates another enhanced integratedcamera/display device that may be employed in the various embodiments.

FIG. 7 illustrates one embodiment of a process flow for operating anintegrated camera display device that is consistent with the variousembodiments presented herein.

FIG. 8 schematically illustrates how users experience an enhancedtelepresence via the various embodiments of videoconferencingapparatuses discussed herein.

FIG. 9 is a block diagram of an example computing device in whichembodiments of the present disclosure may be employed.

DETAILED DESCRIPTION

Briefly stated, various embodiments are directed towards enhancedsystems and methods for providing users with a realistic and naturaltelepresence experience. As used herein, the term “telepresenceexperience” refers to a user's experience when participating in avideo-enable communication session, i.e., the user's experienceassociated with a video conference session (VCS). In the variousembodiments, enhanced arrangements of networked displays and cameras areconfigured to enable a VCS. In an enhanced arrangement of a display andcorresponding camera, a field of view (FOV) of the camera is at leastapproximately aligned with the FOV of the display such that the camera'sFOV is at least approximately equivalent or similar to the display'sFOV. The camera is further configured to not capture image data encodingimages displayed on the corresponding display. By configuring thecamera's FOV to at least approximate the display's FOV, withoutcapturing images of the display, various enhancements are provided overconventional arrangements of cameras and displays that do not includethe alignment of the camera's FOV and the display's FOV. In variousembodiments, the camera's FOV captures at least a substantial portion ofthe display's FOV, such that the camera is not associated with blindspots in front of the display device.

As used herein, the term “camera” refers to any device that senses ordetects electromagnetic (EM) radiation and, in response to the detectedEM radiation, generates image data encoding an image based on thedetected EM radiation. In various embodiments, a camera is a digitalcamera that includes a two-dimensional (2D) array of EM radiation (orphoton) sensors (or detectors), such as but not limited charge-coupleddevices (CCDs). Each discreet sensor of a camera may correspond to oneor more “pixels” of the image data. As such, an image may be encoded indigital image data that includes a 2D array of pixels, where each pixelencodes one or more signals corresponding to the detected EM radiation.The discrete sensors of a camera may be referred to as “the camera'spixels,” or equivalently, “the pixels of the camera.” Thus, dependingupon the content, the term “pixel” may refer to a EM radiation sensor ofa camera or a discrete unit of information or data encoding an image,i.e. the data generated by a photon detector.

As used herein, the terms “display device” and “display” are usedinterchangeably to refer to any device, component, or apparatus that isenabled to display image data. A display device may be a display thatdisplays an image by actively generating light corresponding to theimage data encoding the displayed image. A display may include a monitorfor a computing device, a television display, a touch-sensitive screen,or the like. A display may include, but is not otherwise limited to, acathode ray tube (CRT) display, a liquid crystal display (LCD), a lightemitting diode (LED) display, an organic LED (OLED) display, a plasmadisplay, or the like. Such active display devices may be included in acomputing device, such as but not limited to a mobile device, desktop,or laptop computing device. In other embodiments, a display may includea combination of a projector and surface, such as but not limited to ascreen, i.e. a projection system. In such projection embodiments, aprojector is employed to project an image onto a screen. In any of thevarious projection embodiments, various rear projection techniques maybe employed. That is, the projection and projection screen may beadapted for rear projection techniques. The projection screen may be atleast partially transparent, so that a camera positioned behind theprojection screen may capture images of objects on the front side of theprojection screen. In some embodiments, the camera may be embedded inthe display. Various coatings, such as but not limited toanti-reflection coatings may be applied to one or both sides of theprojection screen to minimize any “ghosting effects,” i.e., a tendencyof the camera to generate image data encoding images projected on theprojection screen.

A display may include a 2D array of discreet regions, referred to aspixels, that each displays a discreet unit of image data, i.e. a pixelof image data. For example, an LCD display may include a 2D array ofdisplay pixels that each displays one or more pixels of image data. Theterms “camera pixel,” “data pixel,” and “display pixel,” may be employedto differentiate the context of the term pixel. Thus, a camera mayinclude a 2D array of camera pixels that generates a 2D array of imagedata pixels, which encode an image captured by the camera. The displaymay display the captured image by employing each display pixel of thedisplay to display one or more of the image data pixels of the imagedata. In projection embodiments, the physical region of the screen thatreflects, or otherwise transmits, the portion of the image thatcorresponds to a discreet image data pixel may be referred to as adisplay pixel.

The terms “the camera's FOV,” or “the FOV of the camera” are usedinterchangeably to indicate the region of space that is opticallyobserved or sensed via the camera. In some embodiments, the camera's FOVindicates one or more regions of the camera's environment from whichelectromagnetic radiation, traversing thru the region and directedtowards the camera, will be detected via the camera. In variousembodiments, each camera pixel of a camera has a FOV, thus a camerapixel's FOV may refer to the region of space that is optically observedor sensed via the camera pixel. A camera's FOV may be equivalent to theunion of the FOV for each of the camera pixels of the camera.

The terms “the display's FOV” or “the FOV of the display” are usedinterchangeably to indicate the region of space from which an imagedisplayed by the display is optically observable via an eye or anothercamera. In some embodiments, the display's FOV indicates one or moreregions of the display's environment from which electromagneticradiation corresponding to the displayed image and generated and/ortransmitted by the display, can be detected via an observer. In variousembodiments, each display pixel of a display has a FOV, thus a displaypixel's FOV may refer to the region of space that the display pixeltransmits EM radiation to. A display's FOV may be equivalent to theunion of the FOV for each of the camera pixels of the camera. In variousembodiments, a camera's FOV, as well as a display's FOV may be indicatedor characterized as an angle or a solid angle.

As indicated above, the various embodiments include enabling a VCS byaligning the FOV of a camera with the FOV of a corresponding display,such that the camera's FOV is at least approximately equivalent orsimilar to the display's FOV. When the camera's and display's FOVs areco-aligned, the FOV of the most of the display pixels of the display areco-aligned with one or more camera pixels of the camera. That is, theoptical line of sight (LOS) of each display pixel is equivalent or atleast similar to the optical LOS of one or more camera pixels. Such aco-alignment of pixels may characterize a mapping or correspondencebetween the display pixels and the camera pixels, wherein the mappinggenerates a correspondence between camera and display pixels withequivalent or similar FOV's.

FIG. 1A schematically illustrates a conventional video conference system150 that a user 158 may employ to participate in a conventional VCS.Conventional system 150 includes a conventional computing device 152that enables a conventional VCS. Conventional computing device 152 isshown from a side profile view in FIG. 1A and includes a display device154 and a front-facing camera (FFC) 156. For example, computing device152 may be a wall mounted or free standing video conferencing system.The FOV of FFC 156 includes the region of space between the hashed linesegments 162 and 164. The FOV of display 154 includes the region ofspace between the dotted line segments 166 and 168. Note that the FOV ofcamera 156 is not aligned with the FOV of the display 154. As such andas schematically shown in FIG. 1A, in conventional system 150, regionsexist within the FOV of display 154 that are outside the FOV of camera156. For example, the hashed region labeled camera's “blind spot” areoutside the FOV of camera 156 but within the FOV of display 154. As usedherein, the term “a camera's blind spot” refer to regions within acorresponding display's FOV but outside the camera's FOV. As shown inFIG. 1A, when portions of the user's 158 body are within the camera'sblind spot, those portions of the user's body will not be imaged via FFC156.

FIG. 1B schematically illustrates an enhanced videoconference system 170that a user 158 may employ to participate in an enhanced VCS. Enhancedsystem 170 includes an enhanced computing device 172 that enables anenhanced VCS. Enhanced computing device 152 is shown from a side profileview in FIG. 1B and includes a display device 174 and a front-facingcamera (FFC) 176 (or simply camera 176).

Similar to conventional computing device 152 of FIG. 1A, enhancedcomputing device 172 may be a wall mounted or free standing videoconferencing system. In other embodiments, enhanced computing device 172may be a desktop computing device, laptop computing device, mobilecomputing device (e.g., a smartphone or tablet), a wearable computingdevice, or any other type of computing device. In FIG. 1B, only displaydevice 174 and FFC 176 are depicted. However, note that enhancedcomputing device may include other components typically included in acomputing device, such as but not limited to a network component. Notethat the physical dimensions of FIG. 1B may not be to scale. Forexample, in the embodiments where enhanced computing device 172 is asmartphone, the physical dimensions of enhanced computing device 172,compared to user 158, would be significantly smaller than thoseillustrated in FIG. 1B. Additionally the ratio of the width to theheight of enhanced computing device 172, as shown in FIG. 1B may beexaggerated, e.g., tablet embodiments. As with other figures herein,FIG. 1B is intended as a schematic representation.

The FOV of FFC 176 includes the region of space between the hashed linesegments 182 and 184. The FOV of display 174 includes the region ofspace between the dotted line segments 186 and 188. In contrast toconventional system 150, the FOV of camera 176 is aligned with the FOVof the display 174. As such and as schematically shown in FIG. 1B, inenhanced system 170, when the camera's FOV is aligned with the display'sFOV, the camera's blind spots are drastically reduced. That is, theamount of space that is within the display's FOV but outside thedisplay's FOV is greatly reduced. As such, as the user 158 approachesdisplay 174, FFC 176 will continue to image the whole of the user's 158body.

Display device 174, as well as any of the display devices discussed inconjunction with FIGS. 1B-9, may be at least partially opticallytransparent display devices. That is, display device 174 may allow atleast partial passage of light from user's 158 side of display device174 to reach camera 176. As such, camera 176 is enabled to generateimage data depicting user 158. Furthermore, as discussed throughout, theFOV of camera 176 is aligned with the FOV of display device 174, camera176 is enabled to generate image data depicting at least a substantialamount of the FOV of the display. Also, camera 176, as well as any ofthe cameras discussed in conjunction with FIGS. 1B-9, are configured tonot generate image data corresponding to images displayed on the displaydevices. In some embodiments, the at least partial transparency of thedisplay devices may be biased in the direction of light passing from theuser to the camera. Various optical films, such as anti-reflectiveand/or polarizing films may be included in any of the display devices toenhance the camera's capability of not generating images displayed onthe display device.

Thus, and in contrast to conventional system 150, when two users areeach employing enhanced systems that include at least partial alignmentbetween a user's display and their camera, the user may participate inan enhanced VCS, where the users' experience is that of almost lookingat each other through a “transparent barrier,” such as a window. Forexample, and similar to a “real world window,” as a user approaches thedisplay, the other user may be provided a visual experience of the userwalking towards them. In contrast, as shown in FIG. 1A, in conventionalVCS systems, as a user approaches their display, they will enter theircamera's blind spot. That is, enhanced system 170 provides a morerealistic telepresence experience for the users.

In the various embodiments, it is contemplated multiple enhanced videoconference systems, such as but not limited to enhanced system 170, maybe remotely positioned, so that remote users may be provided an enhancedtelepresence experience. In addition to business meetings and personalcommunications, such embodiments may be deployed for entertainment andartistic purposes. For a non-limiting example, two art shows may besimultaneously occurring, one in New York and one in Los Angeles. Afirst enhanced system may be installed at the New York art show, while asecond enhanced system may be installed at the Los Angeles art show.Remote performance artists, or other participants of the remote artshows, may interact in novel and interesting ways via a VCS enabled bythe two enhanced systems. Similarly, spectators at remote, butsimultaneously occurring sporting events, may participate inentertaining and realistic interactions. Such enhanced video conferencesystems may be installed in various public spaces, such as but notlimited to public parks, airports, train stations, shopping complexes,restaurants, taverns, pubs, coffee shops, or other meeting places. Suchembodiments may provide fun, entertaining, and amusing one-offanonymized and random interactions between users navigating throughremote public spaces. The embodiments may also be employed in networkedvideo gaming applications. That is, users may participate in networkedvideo games, and interact with each in the video game via theembodiments. At least a portion of the video game environment, as wellas the players (or their virtual avatars) may be displayed on thedisplays, and the other players may interact with displayedplayer/avatar via the enhanced telepresence. The embodiments alsoprovide the capability to capture full-body video data of a user infront on the display/camera system. That is, full-body video data may becaptured of the user, as the user approaches the front of the display.As a user approaches a conventional camera, with a limited FOV, at leastportions of the user's body may fall outside of the camera's FOV. Thus,the embodiments may be applied to capture and archive full-body videodata of a user directly in front of a display device. Unless stated tothe contrary, the physical dimensions depicted in the figures presentedherein need not be to scale. That is, the figures are intended asschematic representation that illustrate various embodiments. Forexample, as noted above, in FIG. 1C, enhanced computing device 172 maybe a tablet, and thus significantly smaller than user 158. In otherembodiments, the relative sizing of enhanced computing device 172 anduser 158 may be approximated in FIG. 1C.

An Exemplary Enhanced Videoconference System

FIG. 1C illustrates an exemplary embodiment of an enhancedvideoconference system 100 that is enabled to provide users with arealistic and natural telepresence experience that is consistent withthe various embodiments discussed herein. System 100 is a distributedenhanced videoconference system. System 100 includes a first enhancedvideoconferencing apparatus (VCA) 120, located at location A, and asecond enhanced VCA 130, located at location B. The first VCA 120 andthe second VCA 130 are communicatively coupled via communication network110.

First VCA 120 includes at least a first display device 126, a firstcamera, and a first network component. Similarly, second VCA 130includes at least a second display device 136, a second camera, and asecond network component. For simplicity, the first/second cameras, aswell as the first/second network components are not shown in FIG. 1C.The first and second network components enable transmission of data,including at least image data, via communication network 110.

In FIG. 1C, the first user 122 and second user 132 employ first VCA 120and second VCA 130, respectively, to participate in a real-time, or nearreal-time, enhanced videoconferencing session (VCS). Location A andlocation B may be remote from one another at least because of thecommunicatively coupling of the first VCA 120 and the second VCA 130,via communication network 110. Thus, the first user 122 may be remotefrom the second user 132. Although only two users (the first user 122and the second user 132) are shown in in FIG. 1C, it should beunderstood that more than two users may participate in the enhanced VCSvia the communicatively coupling of additional VCAs via thecommunication network 110.

Various embodiments of an enhanced VCA, such as but not limited to firstVCA 120 and second VCA 130 are discussed at least in conjunction FIG.2-9. However, briefly here, each of first and second VCAs 120/130 may beat least somewhat similar to enhanced computing device 170 of FIG. 1B.For example, with regard to first VCA 120, the FOV of the first camerais aligned with the FOV of the first display device 126, similar to theco-alignment of camera 176 and display 174 of FIG. 1B. Likewise, withregard to second VCA 130, the FOV of the second camera is aligned withthe FOV of the second display device 136 similar to the co-alignment ofcamera 176 and display 174. As discussed in the conjunction with at FIG.1B, co-alignment of the respective FOV of a display and a camera provideenhanced telepresence for users 122 and 132.

As shown in FIG. 1C, the first camera of the first VCA 120 generatesvideo data encoding images of the first user 122. The video dataencoding images of the first user 122 is transmitted, via communicationnetwork 110, to the second VCA 130. The video data of the first user 122is displayed via the second display device of the second VCA 130, asillustrated as image 134. Similarly, the second camera of the second VCA130 generates video data encoding images of the second user 132. Thevideo data encoding images of the second user 132 is transmitted, viacommunication network 110, to the first VCA 120. The video data of thesecond user 132 is displayed via the first display device of the firstVCA 120, as illustrated as image 124. First VCA 120 and second VCA 130may transmit and receive image data to/from communication network 110via network components included in VCAs 120/130.

Accordingly, video data captured and/or generated by the first camera iswhat is viewable by the second user 132 and video data captured and/orgenerated by the second camera is what is viewable by the first user122. When a user is participating in a VCA, the first user 122“experiences” the “telepresence” of the second user 132 via the videodata captured and/or generated via the second camera (the second videodata) of the second VCA 130 and the corresponding display of this secondvideo displayed via the first display device of the first VCA 120.Likewise, the second user 132 experiences the telepresence of the firstuser 122 via the video data captured and/or generated via the firstcamera (the first video data) of the first VCA 120 and the correspondingdisplay of this first video displayed via the second display device ofthe second VCA 130.

Because of the mutual alignment of the FOV of the first camera and theFOV of first display 126, the first user 122 may approach first display126 without moving into a blind spot of the first camera. Thus, theimage 134 rendered on second display 136 will continue to include thewhole presence of the first user 122 as the first user 122 approachesthe first display 126. Likewise, the second user 124 may approach seconddisplay 136 without moving into a blind spot of the second camera. Thus,the image 124 rendered on first display 126 will continue to include thewhole presence of the second user 132 as the second user 132 approachesthe second display 136.

System 100 may further include one or more computing devices, such asbut not limited to computer device 102. Such computing devices may becommunicatively coupled to at least one of the VCAs 120 or 130 viacommunication network 110. An exemplary, but non-limiting embodiment ofa computing device is discussed in conjunction with at least computingdevice 500 of FIG. 5. However briefly, a computing device may bereferred to as a processor device because a computing device generallyemploys and/or includes a processor.

Communication network 110 may be any communication network, includingvirtually any wired and or wireless communication technologies, wiredand/or wireless communication protocols, and the like. It should beunderstood that communication network 110 may be virtually anycommunication network that communicatively couples a plurality ofcomputing devices in such a way as to enable users of computing devicesto exchange information via the computing devices.

An Exemplary Enhanced Videoconferencing Apparatus

FIG. 2 schematically illustrates an exemplary embodiment of an enhancedvideoconferencing apparatus (VCA) 200 that may be employed in theenhanced systems 170 and 100 of FIGS. 1B-1C. For instance, VCA 200 maybe similar to at least one of enhanced computing device 170 of FIG. 1Bor first VCA 120 and/or second VCA 130 of FIG. 1C. VCA 200 includes atransparent, or at a least semi-transparent, display device 210. Forinstance, display device 210 may include transparent, or at leastsemi-transparent, a two-dimensional arrays of light-emitting displaypixels. For instance, the transparent display pixels of display device210 may be liquid crystal (LCD) pixels, light-emitting diode (LED)display pixels, or the like. Display device 210 may be a transparentorganic LED (OLED) display device.

VCA 200 includes a photon-detector, such as camera 230. Because displaydevice 210 is transparent, or at least semi-transparent, camera 230 maycapture and/or generate image data (such as video image data) of objectson the opposite side (i.e., the external side) of display device 210,such as user 202. VCA may include one or more light sources 240 that mayat least partially illuminate the objects positioned on the externalside of display device 210. As shown in FIG. 2 via the hashed lines,camera 230 and display device 210 may include mutually aligned pixels.Optical subsystem 220 may at least partially enable the mutual alignmentof the camera pixels of camera 230 and the display pixels of displaydevice 210. Optical subsystem 220 may include various optical elements,such as lenses, mirrors, optical fibers, shutters, apertures,diffraction gratings, and the like to enable attenuating, minimizing (orat least decreasing ghosting effect), and increasing/enhancing themutual alignment of the camera pixels of camera 230 and the transparentdisplay pixels of display device 210.

Note that in some embodiments, the mutual alignment of the camera pixelsof camera 230 and the display pixels of display device 210 may include amapping or correspondence between the camera pixels and the displaypixels. In at least one non-limiting embodiment, the mapping may be aone-to-one mapping. For instance, each camera pixel may be positionedbehind (or in front of) the corresponding display pixel. In otherembodiments, the mapping may not be a one-to-one mapping. The displaypixel may be at least a partially transparent display pixel. The camerapixels may be smaller than the corresponding display pixels, and theoptical subsystem 220 enables the “zoom-out” of the camera pixels, suchthat the “area” or solid angle subtended by a camera pixel is equivalentor similar to the area or solid angle of the corresponding pixel. In atleast some embodiments, the camera pixel and the display may be one inthe same. That is, a pixel may be enabled to be both photo-sensitive(i.e., a camera pixel) as well as an emitter of photons (i.e., a displaypixel). In some embodiments, the pixel may be transitioned periodicallyto a display mode and a camera mode. For instance, the mode of suchdual-purposes pixel may be modulated at 60 Hz, where the pixels takes 60frames of image data of the user per second, and alternately display 60frames of image data to the user.

VCA 200 may include a computing 250. Various embodiments of computingdevices are discussed in conjunction with at least computing device 500of FIG. 5. However, computing device 250 may include a network device tocommunicatively couple VCA 200 to a communication network, such ascommunication network 110 of FIG. 1C. Computing device 250 may includeone or more processing devices to process and/or buffer at least some ofthe video data captured by camera 230 or the video data received via thecommunication network. VCA 200 may include a communication bus 270 tocommunicatively couple the various components.

The various components of VCA 200 may be housed within a light-tighthousing 280, where the only sources of light entering into housing 280are through the transparent display device 210 or any apertures and/orlenses for projecting light from light source 240. In some embodimentslight source 240 may be located outside of housing 280 to minimize lightentering into housing 280. Minimizing external light impinging on thepixels of camera 230 may minimize ghosting issues. Herein, ghostingrefers to feedback event, where at least due to the transparency ofdisplay device 210, the camera 230 may generate video data correspondingto the image displayed via display device 210. Minimizing the externallight exposing the photon-detecting pixels of camera 230 may helpminimize such ghosting feedback.

Anti-reflective elements, such as anti-reflective film 260, located onthe external side of the display device 210 may reduce the reflectionand ghosting issues. Additional optical components such as polarizedlenses may additionally be placed on the external side of display deviceto further decrease ghosting issues. The illumination of the user 202via light source 240 may also reduce ghosting effects. Staggering ashuttering of the camera 230 and the display 210 may further reduceghosting. For instance, a phase difference of it may be introduced suchthat camera 230 is acquiring video data only when display device is notdisplaying and vide versa. A shuttering frequency may be chosen, suchthat a flickering is not observable via the users.

As shown in FIG. 2, the size of display device 210 may be that on theorder of the size of the user 202, or may be smaller, such as a sizemore consistent with desktop display monitors, or even the display ofmobile devices, such as tablets and smartphones. Note that in someembodiments, the elements of VCA may be integrated into a form factorthat approximates a flat panel display. For various exemplary, butnon-limiting embodiments, the ratio of the physical width to thephysical height of VCA 200 may be significantly exaggerated asillustrated in FIG. 2.

FIG. 3A schematically illustrates a video conferencing session (VCS)between users that is enabled via an enhanced videoconference system 300that is consistent with the various embodiments herein. System 300includes a first enhanced videoconferencing apparatus (VCA) 310 and asecond enhanced VCA 330 that are communicatively coupled viacommunication network 110. A first user 318 employs the first VCA 310and a second user 338 employs to second VCA 330 to participate in anenhanced VCS. The vertical hashed line indicates that the first user 318and the second 338 may be positioned in remote locations. As anexemplary non-limiting embodiment, first VCA 310 may be installed at afirst art show in New York and second VCA 330 may be installed at asecond art show in Los Angeles, where the art shows are occurringsimultaneously. First and second users 318/338 may be participants orperformers in the respective art shows.

First VCA 310 includes a first projection screen 312 and a firstenhanced computing device 314 that is communicatively coupled tocommunication network 110. First projection screen 312 may be at least apartially transparent screen. In some embodiments, first projectionscreen 312 may be preferentially transparent in one direction, asopposed to the other direction. First computing device 314 includes afirst integrated camera/projector component 316 that includes a firstcamera and a first image projector. Various optical component includedin first integrated camera/projector component 314 may at leastpartially align the FOV of the first camera with the FOV of the firstimage projector, as discussed herein.

Likewise, second VCA 330 includes a second projection screen 332 and asecond enhanced computing device 334 that is communicatively coupled tocommunication network 110. Similar to first projection screen 312,second projection screen 332 may be at least a partially transparentscreen. In some embodiments, second projection screen 332 may bepreferentially transparent in one direction, as opposed to the otherdirection. Similar to first computing device 314, second computingdevice 334 includes a second integrated camera/projector component 336that includes a second camera and a second image projector. Variousoptical component included in second integrated camera/projectorcomponent 334 may at least partially align the FOV of the second camerawith the FOV of the second image projector, as discussed herein.

Because first projection screen 312 is at least partially transparent,the first camera of first computing device 314 may capture first imagedata, including video image data, of the first user 318. Likewise, thesecond camera of second computing device 334 may capture second imagedata of the second user 338. Via the communication network, the firstimage data may be provided to the second computing device 334 and thesecond image data may be provided to the first computing device 314. Thefirst image projector included in the first integrated camera/projectorcomponent 316 may project the second image data onto first projectionscreen 312. Such a projection results in a display of projected image328 of the second user 338 on the first projection screen 312.Similarly, the second image projector included in the second integratedcamera/projector component 336 may project the first image data ontosecond projection screen 332. As shown in FIG. 3A, this projectionresults in a display of projected image 348 of the first user 318 on thesecond projection screen 332.

As shown in FIG. 3A, and because of the co-alignment of the camera andprojector/screen FOVs, first user 318 may approach the first projectionscreen 312 arbitrarily close, and the entire image 348 of first user 318is projected onto the second projection screen 332. Similarly, seconduser 338 may approach the second projection screen 332 withouttravelling into a blind spot of the second camera included in the secondintegrated camera/projector component 336.

FIG. 3B schematically illustrates another embodiment of an enhancedvideoconferencing apparatus (VCA) 360 that may be employed in system 300of FIG. 3A. In FIG. 3B, first user 318 is employing VCA 360 toparticipate in the VCS of FIG. 3A. VCA 360 includes a display device362. Display device 362 may be similar to display device 210 of FIG. 2,in that display device 362 may be at least partially transparent. Thetransparent display pixels of display device 362 may be liquid crystal(LCD) pixels, light-emitting diode (LED) display pixels, or the like.Display device 362 may be a transparent organic LED (OLED) displaydevice. VCA 360 also includes an integrated camera 364 that is enabledto capture image data of first user 318.

In some embodiments, camera 364 includes integrated optics that alignthe FOV of camera 364 with FOV of display device 360. In otherembodiments, the optical components may be embedded within displaydevice 262. As with other embodiments herein, because of theco-alignment with camera's 364 FOV and display device's 362 FOV, thereare minimal camera blind spots within the FOV of display device 362.

Generalized Processes for an Enhanced Telepresence Experience

Process 400 of FIG. 4 will now be discussed. Briefly, process 400 may beemployed to provide enhanced experiences for users participatingvideoconferencing sessions (VCS). Process 400 is consistent with thevarious embodiments discussed herein. Process 400 may be implemented,executed, or otherwise performed via a single and/or a combination ofthe various embodiments discussed herein. For example, system 170 ofFIG. 1B, system 100 of FIG. 1C, or system 300 of FIG. 3A may enable suchprocesses. In various embodiments, computing devices, such as but notlimited to user-computing device 102 of FIG. 1C, computing device 250 ofFIG. 2, or computing device 900 of FIG. 9 are enabled to carry out atleast portions of processes 400. In some embodiments, avideoconferencing apparatus (VCA), such as any of those discussed hereinand shown in accompanying figures may enable at least portions ofprocesses 400.

FIG. 4 illustrates one embodiment of a process flow for providingenhanced telepresence experiences that are consistent with the variousembodiments presented herein. Process 400 begins, after a start block,at block 402 where the FOV of a first camera and the FOV of a firstdisplay device of a first VCA are mutually aligned. In at least someembodiments, the FOV of at least a portion of the camera pixels includedin the first camera are mutually aligned with the FOV of at least aportion of the display pixels in the first display device. At block 404,the FOV of a second camera and the FOV of a second display device of asecond VCA are mutually aligned. At block 406, the first and second VCAsare communicatively coupled via a communication network.

At block 408, a videoconferencing session (VCS) is initiated between afirst user employing the first VCA and a second user employing thesecond VCA via the communicative coupling between the first VCA and thesecond VCA. At block 410, the first camera may generate and/or capturevideo image data of the first user. Likewise, the second camera maygenerate and/or capture video image data of the second user. At block412, the video data of the first user may be transmitted to the secondVCA via the initiated VCS. Similarly, the video data of the second usermay be transmitted to the first VCA via the VCS.

At block 414, the video data of the second user may be received at thefirst VCA and the video data of the first user may be received at thesecond VCA. At block 416, the first display device may display the videodata of the second user. The second display device may display the videodata of the first user. Process 400 may terminate and/or return acalling process.

Additional Embodiments for Aligning Field of Views of a Camera and aDisplay Device

FIG. 5A shows a schematic view of projection-based enhancedvideoconferencing apparatus (VCA) 510 that is consistent with thevarious embodiments. User 518 may employ VCA 510 to participate in avideo conferencing sessions (VCS), as discussed herein. VCA 510 mayinclude features similar to first and/or second VCAs 310/330 of FIG. 3A.For example, VCA 510 include projection screen 512. Rather thanemploying an integrated camera/projector component, such as integratedcamera projector components 316/338 of FIG. 3A, VCA 510 includes adiscreet camera 514 and a discreet projector 516. The projector 522 andthe camera may be aligned at least slightly off-axis, as shown in FIG.5A. On the back side of projection screen 512, the FOV of camera 514 isindicated by hashed lines 526 and 528. On the backside of projectionscreen 512, the FOV of projector 526 is indicated by hashed lines 522and 524. On the front side of projection screen 512, the FOV of camera516 and FOV of projector 516 are at least in partial alignment, asindicated via hashed lines 532 and 534. Thus, similar to otherembodiments discussed herein, camera's 514 blind spots in the FOV ofscreen 512 are reduced and/or minimized. In some embodiments, opticalcomponents integrated into camera 514 and/or projector 516 may beemployed to increase the mutual alignment of the FOV of camera 514 andthe FOV of projector 516. Some embodiments of VCA 510 may include alight source 542 that is configured and arranged to illuminate at leasta significant portion of FOV of camera 514 and the co-aligned FOV ofscreen 512. Additional optical system, not shown in FIG. 5A, may beincluded to focus the light emitting from light source 542 to particularportions of the FOV of camera 514. The dotted lines 536 and 538 indicatethe illumination filed of light source 542. The intensity of lightsource 542 may be modulated to reduce the propensity of camera 514capturing images projected onto projection screen 512. Thus, lightsource 542 may be modulated to reduce any “ghosting” effects for theimage data captured via camera 514.

VCA 510, as well as any other projection-based VCA discussed herein,including but not limited to first VCA 310 and second VCA 330 of FIG. 3Amay be housed in a light tight, or at least semi-light tight housing,such as housing 540. Housing 540 may house at least camera 514 andprojector 516, as well as other components, such as a networkcommunication module. Note that the display surface of screen 512 facestowards users 538. At least a portion of projection screen may beincluded in housing 540. In some embodiments, light source 542 may behoused within housing 540. In such embodiments, housing may include anaperture to allow passage of the light transmitted from light source 542to the external environment of housing 540. In other embodiments, lightsource may be positioned external to housing 540.

FIG. 5B schematically illustrates another embodiment of an enhancedvideoconferencing apparatus (VCA) 560 that is consistent with thevarious embodiments herein. User 568 may employ VCA 560 to participatein a video conferencing sessions (VCS), as discussed herein. VCA 560includes a display device 562. Display device 562 may be similar todisplay device 362 of FIG. 3B, in that display device 562 may be atleast partially transparent. The transparent display pixels of displaydevice 362 may be liquid crystal (LCD) pixels, light-emitting diode(LED) display pixels, or the like. Display device 362 may be atransparent organic LED (OLED) display device. VCA 560 also includes anintegrated array of multiple cameras (i.e., camera array 564) that areenabled to capture image data of first user 568.

In VCA 560, the FOV for each camera included camera array 564 is alignedwith a portion of the FOV of display 562. In various embodiments, theunion of all the FOVs of the cameras included in camera array 564 is atleast approximately aligned with the total FOV of display 562. Thecameras of camera array 562 may be operated simultaneously to captureimage data of user 568. The image data from all the cameras may becombined to generate an image that includes a FOV that approximates theFOV of display 562. Thus, the total FOV of camera array 562 at leastapproximates the FOV of display 562. Accordingly, VCA 560 may beemployed to provide the enhanced VCS as discussed herein.

FIG. 6A schematically illustrates an enhanced integrated camera/display(ICD) device 600 that may be employed in the various embodiments. ICDdevice 600 includes both camera pixels and display pixels. In one nonlimited embodiment, ICD device 600 includes vertical columns of camerapixels, such as but not limited to vertical column 602 of camera pixels,alternating with vertical columns of display pixels, such as but notlimited to vertical column 604 of display pixels. The camera pixels mayinclude, but are not limited to charge-coupled device (CCD) pixels,complementary metal-oxide-semiconductor (CMOS) pixels, or the like. Thedisplay pixels may include but are not limited to liquid crystal display(LCD) pixels, light emitting diode (LED) pixels, organic LED (OLED)pixels, and the like. In other embodiments not shown in FIG. 6A, an ICDdevice may include alternating horizontal rows of camera and displaypixels.

Thus, ICD device 600 includes a camera comprising the plurality of rowsor columns of camera pixels. ICD device 600 also includes a displaycomprising the plurality of rows or columns of display pixels.Accordingly, in a VCS, such as those discussed herein, via the rows orcolumns of display pixels of the integrated display, ICD device 600 maydisplay image data received from other devices. ICD device 600 may alsocapture image data of a user via the rows or columns of camera pixels ofthe integrate camera. Because of the alternating nature of the rows orcolumns of camera and display pixels, the FOVs of the integrated displayand camera are aligned. Similar to the other embodiments discussedherein, ICD device 600 may be employed to enabled a VCS that does notinclude camera blind spots in front of the display.

In various embodiments, the density of alternating rows or columns ofdisplay and camera pixels is sufficient such that a human eye cannotdetect the interleaving of rows or columns of display and camera pixels.That is, a human eye would see a high resolution image displayed via theintegrated display, while a high resolution image data of them iscaptured via the integrated camera. The resolution of the captured imagedata is sufficient, such that when the high resolution image data isdisplayed on another display device, a human user cannot detect the“gaps” between the rows or columns of image data.

As noted above, in some embodiments, the display and camera pixels areoperated simultaneously. In other embodiments, the display and camerapixels are alternating in time. For example, time may be discretized orbucketed via an operation frequency. In a first instance of discretizedtime, the display pixels may display a single frame of video image data.In the next instance of discretized time, the camera pixels may capturea single frame of video image data. In the following instance ofdiscretized time, the display pixels are refreshed via the next frame ofvideo image data to be displayed. In a non-limiting example, each of thecamera and display pixels may be operated at 100 Hz, such that thedisplay pixels display 100 frames of video image data per second and thecamera pixels capture 100 frames of video image data per second. In thisnon-limiting example, a t=0.000, the display pixels display a firstframe of image data. At t=0.005, the camera pixels capture a secondframe of image data and at t=0.010, the display pixels display a thirdframe of image data. In such embodiments, the operating frequencies ofthe display and camera pixels are chosen such that a human user wouldnot detect the alternating operation of the display of the image dataand the capture of the image data.

FIG. 6B schematically illustrates another enhanced integratedcamera/display (ICD) device 650 that may be employed in the variousembodiments. The features and operation of ICD device 650 may be similarto that of ICD device 600, rather than interleaved rows or columns ofdisplay and camera pixels, a checkerboard pattern of alternating displayand camera pixels is constructed. For example, as shown in FIG. 6B, thecheckerboard pattern of ICD device 650 includes a camera pixel 652adjacent a display pixel 654.

FIG. 7 illustrates one embodiment of a process flow for operating anintegrated camera display device (ICD) that is consistent with thevarious embodiments presented herein. For example, process 700 may beemployed to operate ICD device 600 of FIG. 6A or ICD device 650 of FIG.6B, in alternating the operation of the camera and display pixels. Itshould be noted process 700 may be employed to operated anycorresponding pair of displays and camera devices discussed herein inthe service of enabling the enhanced VCS. At block 702, the camerapixels are operated to capture a first frame of image data encoding animage. At block 704, the first frame of image data is provided to asecond display device. At block 706, a second frame of image dataencoding a second image is received. At block 708, display pixels of afirst display are operated to display the received second frame of imagedata.

FIG. 8 schematically illustrates how users experience an enhancedtelepresence via the various embodiments of videoconferencingapparatuses discussed herein. System 800 includes a first VCA 810 in afirst remote location and a second VCA 830 in a second remote location.The first and second VCAs 810/830 are communicatively coupled via acommunication network, not depicted in FIG. 8. First user 818 isemploying first VCA 810 and second user 838 is employing second VCA 830to participate in an enhanced VCS, as discussed herein via thecommunicatively coupling of first and second VCAs 810/830. In someembodiments, first and second VCAs 810/830 may be projection-based VCAs,such as but not limited to any of VCAs 310, 330, or 510 of FIGS. 3A and5A.

First VCA 810 is housed within at least a partially light tight firsthousing 840, while second VCA is housed within at least a partiallylight tight second housing 842. An image 828 of second user 838 is rearprojected (via a projector housed within first housing 840) onto aprojection screen of first VCA 810. The image data encoding image 828 ishoused with second housing 842 and provided to the projector housedwithin first housing 840 via the communication network. Similarly, animage 848 of the first user 818 is rear projected onto a projectionscreen of the second VCA 830, via a projector housed within second VCA830. The image data displaying image 848 was generated by a camerahoused in first housing 840.

A close-up 850 is provided that shows the second user 838 interactingwith the image 848 of first user 818. At least due to the mutualalignment of the FOVs of the cameras and projections screens of therespective first and second VCAs 810/830, and the resulting lack ofcamera blind spots in front of (i.e., the user's side) of the projectionscreens, users 818/838 are provided the enhanced telepresence experienceof interacting with each other through a transparent barrier 852, asdepicted in close-up 850.

As discussed throughout, and depicted visually in close-up 850, anenhanced telepresence experience may include the simulation that theusers (or participants) of a VCS are interacting with one anotherthrough a transparent barrier 852, such as a glass window. Thus, theusers 818/838 may experience “hand-to-hand” interactions, such as shownin close-up 850 of FIG. 8. That is, each user may touch a correspondinglocation on their individual location and receive visual feedback thatthe other user is touching their display at the same location. Thus, theuser's experience a simulation of interactions through a transparentmedium, such as, but not limited to, the above described “hand-to-hand”interaction shown in close-up FIG. 850. Note that the users 818/838 donot have to actually touch their displays. Rather, a similar effectresults when the users position a portion of their body, such as a foot,hand, or other extension in close proximity to the display.

Illustrative Computing Device

Having described embodiments of the present invention, an exampleoperating environment in which embodiments of the present invention maybe implemented is described below in order to provide a general contextfor various aspects of the present invention. Referring to FIG. 9, anillustrative operating environment for implementing embodiments of thepresent invention is shown and designated generally as computing device900. Computing device 900 is but one example of a suitable computingenvironment and is not intended to suggest any limitation as to thescope of use or functionality of the invention. Neither should thecomputing device 900 be interpreted as having any dependency orrequirement relating to any one or combination of componentsillustrated.

Embodiments of the invention may be described in the general context ofcomputer code or machine-useable instructions, includingcomputer-executable instructions such as program modules, being executedby a computer or other machine, such as a smartphone or other handhelddevice. Generally, program modules, or engines, including routines,programs, objects, components, data structures, etc., refer to code thatperform particular tasks or implement particular abstract data types.Embodiments of the invention may be practiced in a variety of systemconfigurations, including hand-held devices, consumer electronics,general-purpose computers, more specialized computing devices, etc.Embodiments of the invention may also be practiced in distributedcomputing environments where tasks are performed by remote-processingdevices that are linked through a communications network.

With reference to FIG. 9, computing device 900 includes a bus 910 thatdirectly or indirectly couples the following devices: memory 912, one ormore processors 914, one or more presentation components 916,input/output ports 918, input/output components 920, and an illustrativepower supply 922. Bus 910 represents what may be one or more busses(such as an address bus, data bus, or combination thereof). Although thevarious blocks of FIG. 9 are shown with clearly delineated lines for thesake of clarity, in reality, such delineations are not so clear andthese lines may overlap. For example, one may consider a presentationcomponent such as a display device to be an I/O component, as well.Also, processors generally have memory in the form of cache. Werecognize that such is the nature of the art, and reiterate that thediagram of FIG. 9 is merely illustrative of an example computing devicethat can be used in connection with one or more embodiments of thepresent disclosure. Distinction is not made between such categories as“workstation,” “server,” “laptop,” “hand-held device,” etc., as all arecontemplated within the scope of FIG. 9 and reference to “computingdevice.”

Computing device 900 typically includes a variety of computer-readablemedia. Computer-readable media can be any available media that can beaccessed by computing device 900 and includes both volatile andnonvolatile media, removable and non-removable media. By way of example,and not limitation, computer-readable media may comprise computerstorage media and communication media.

Computer storage media include volatile and nonvolatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer-readable instructions, data structures,program modules or other data. Computer storage media includes, but isnot limited to, RAM, ROM, EEPROM, flash memory or other memorytechnology, CD-ROM, digital versatile disks (DVD) or other optical diskstorage, magnetic cassettes, magnetic tape, magnetic disk storage orother magnetic storage devices, or any other medium which can be used tostore the desired information and which can be accessed by computingdevice 900. Computer storage media excludes signals per se.

Communication media typically embodies computer-readable instructions,data structures, program modules or other data in a modulated datasignal such as a carrier wave or other transport mechanism and includesany information delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media includes wired media such as awired network or direct-wired connection, and wireless media such asacoustic, RF, infrared and other wireless media. Combinations of any ofthe above should also be included within the scope of computer-readablemedia.

Memory 912 includes computer storage media in the form of volatileand/or nonvolatile memory. Memory 912 may be non-transitory memory. Asdepicted, memory 912 includes instructions 924. Instructions 924, whenexecuted by processor(s) 914 are configured to cause the computingdevice to perform any of the operations described herein, in referenceto the above discussed figures, or to implement any program modulesdescribed herein. The memory may be removable, non-removable, or acombination thereof. Illustrative hardware devices include solid-statememory, hard drives, optical-disc drives, etc. Computing device 900includes one or more processors that read data from various entitiessuch as memory 912 or I/O components 920. Presentation component(s) 916present data indications to a user or other device. Illustrativepresentation components include a display device, speaker, printingcomponent, vibrating component, etc.

I/O ports 918 allow computing device 900 to be logically coupled toother devices including I/O components 920, some of which may be builtin. Illustrative components include a microphone, joystick, game pad,satellite dish, scanner, printer, wireless device, etc.

Embodiments presented herein have been described in relation toparticular embodiments which are intended in all respects to beillustrative rather than restrictive. Alternative embodiments willbecome apparent to those of ordinary skill in the art to which thepresent disclosure pertains without departing from its scope.

From the foregoing, it will be seen that this disclosure in one welladapted to attain all the ends and objects hereinabove set forthtogether with other advantages which are obvious and which are inherentto the structure.

It will be understood that certain features and sub-combinations are ofutility and may be employed without reference to other features orsub-combinations. This is contemplated by and is within the scope of theclaims.

In the preceding detailed description, reference is made to theaccompanying drawings which form a part hereof wherein like numeralsdesignate like parts throughout, and in which is shown, by way ofillustration, embodiments that may be practiced. It is to be understoodthat other embodiments may be utilized and structural or logical changesmay be made without departing from the scope of the present disclosure.Therefore, the preceding detailed description is not to be taken in alimiting sense, and the scope of embodiments is defined by the appendedclaims and their equivalents.

Various aspects of the illustrative embodiments have been describedusing terms commonly employed by those skilled in the art to convey thesubstance of their work to others skilled in the art. However, it willbe apparent to those skilled in the art that alternate embodiments maybe practiced with only some of the described aspects. For purposes ofexplanation, specific numbers, materials, and configurations are setforth in order to provide a thorough understanding of the illustrativeembodiments. However, it will be apparent to one skilled in the art thatalternate embodiments may be practiced without the specific details. Inother instances, well-known features have been omitted or simplified inorder not to obscure the illustrative embodiments.

Various operations have been described as multiple discrete operations,in turn, in a manner that is most helpful in understanding theillustrative embodiments; however, the order of description should notbe construed as to imply that these operations are necessarily orderdependent. In particular, these operations need not be performed in theorder of presentation. Further, descriptions of operations as separateoperations should not be construed as requiring that the operations benecessarily performed independently and/or by separate entities.Descriptions of entities and/or modules as separate modules shouldlikewise not be construed as requiring that the modules be separateand/or perform separate operations. In various embodiments, illustratedand/or described operations, entities, data, and/or modules may bemerged, broken into further sub-parts, and/or omitted.

The phrase “in one embodiment” or “in an embodiment” is used repeatedly.The phrase generally does not refer to the same embodiment; however, itmay. The terms “comprising,” “having,” and “including” are synonymous,unless the context dictates otherwise. The phrase “A/B” means “A or B.”The phrase “A and/or B” means “(A), (B), or (A and B).” The phrase “atleast one of A, B and C” means “(A), (B), (C), (A and B), (A and C), (Band C) or (A, B and C).”

What is claimed is:
 1. A method for providing an enhanced telepresence, comprising: initiating a videoconference session between a first video conference apparatus (VCA) and a second VCA, the first and second VCAs each comprising an integrated display panel having uniformly interleaved display pixels and camera pixels, wherein each camera pixel of the first VCA corresponds to a display pixel of the second VCA that is physically larger than the corresponding camera pixel of the first VCA; employing one or more optical components to match a field of view of each of the camera pixels of the first VCA with a field of view of each of the corresponding display pixels of the second VCA by increasing a solid angle subtended by each of the camera pixels of the first VCA to match a corresponding solid angle of each of the corresponding display pixels of the second VCA; generating first image data from the first VCA; providing the first image data to the second VCA to be displayed on the integrated display panel of the second VCA; and receiving, by the first VCA, second image data from the second VCA to be displayed on the integrated display panel of the first VCA.
 2. The method of claim 1, wherein the integrated display panel includes a light emitting diode (LED) panel comprising alternating rows or columns of camera pixels and display pixels.
 3. The method of claim 1, further comprising displaying a virtual avatar on the integrated display panel, wherein image data for the virtual avatar is generated based on the second image data from the second VCA.
 4. The method of claim 1, wherein the camera pixels are positioned such that a union of a FOV of each of the camera pixels substantially covers a FOV of the integrated display panel.
 5. The method of claim 1, wherein the integrated display panel comprises camera pixels and display pixels arranged into a checkered pattern, wherein the camera pixels and the display pixels do not overlap.
 6. The method of claim 1, wherein each of the camera pixels and the display pixels are mapped to a coordinate position, wherein image data is encoded with position data, wherein mapped pixels create a correspondence between camera pixels and display pixels with a similar FOV.
 7. The method of claim 5, wherein the camera pixels and the display pixels operate simultaneously.
 8. An apparatus for providing enhanced telepresence experiences, the apparatus comprising: an integrated display panel having uniformly interleaved display pixels and camera pixels, wherein each camera pixel of the integrated display panel corresponds to a display pixel of a computing device that is external to the apparatus that is physically larger than the corresponding camera pixel of the integrated display panel; the integrated display device enabled to employ one or more optical components to match a field of view of each of the camera pixels of the integrated display panel with a field of view of each of the corresponding display pixels of the computing device that is external to the apparatus by increasing a solid angle subtended by each of the camera pixels of the integrated display panel to match a corresponding solid angle of each of the corresponding display pixels of the computing device that is external to the apparatus; a network device enabled to receive first image data from the computing device that is external to the apparatus and transmit second image data generated by the integrated display panel, by the camera pixels, to the computing device that is external to the apparatus; and a processor device that is enabled to display images encoded in the received first image data on the display device via the display pixels.
 9. The apparatus of claim 8, wherein the integrated display panel includes a light emitting diode (LED) panel comprising alternating rows or columns of camera pixels and display pixels.
 10. The apparatus of claim 8, further comprising displaying a virtual environment on the integrated display panel.
 11. The apparatus of claim 8, wherein the camera pixels are positioned such that a union of a FOV of each of the camera pixels substantially covers a FOV of the integrated display panel.
 12. The apparatus of claim 8, wherein the integrated display panel comprises camera pixels and display pixels arranged into a checkered pattern, wherein the camera pixels and the display pixels do not overlap.
 13. The apparatus of claim 8, wherein each of the camera pixels and the display pixels are mapped to a coordinate position, wherein image data is encoded with position data, wherein mapped pixels create a correspondence between camera pixels and display pixels with a similar FOV.
 14. The apparatus of claim 12, wherein the camera pixels and the display pixels operate simultaneously.
 15. One or more non-transitory computer readable storage medium storing computer-useable instructions that, when executed by one or more computing devices, causes the one or more computing devices to perform operations for providing an enhanced telepresence, the operations comprising: generating first image data via an integrated display panel having uniformly interleaved display pixels and camera pixels, wherein each camera pixel of the integrated display panel corresponds to a display pixel of a computing device that is external to the integrated display panel that is physically larger than the corresponding camera pixel of the integrated display panel; employing one or more optical components to match a field of view of each of the camera pixels of the integrated display panel with a field of view of each of the corresponding display pixels of the computing device that is external to the integrated display panel by increasing a solid angle subtended by each of the camera pixels of the integrated display panel to match a corresponding solid angle of each of the corresponding display pixels of the computing device that is external to the integrated display panel; receiving second image data at the integrated display panel, wherein the second image data is encoded to include position data; and displaying the second image data via the display pixels of the integrated display panel.
 16. The computer readable storage medium of claim 15, wherein the integrated display panel includes a transparent liquid crystal display (LCD) panel comprising each of the display pixels positioned adjacent to at least one camera pixel.
 17. The computer readable storage medium of claim 15, the operations further comprising displaying a virtual avatar on the integrated display panel, wherein image data for the virtual avatar is generated based on the received second image data.
 18. The computer readable storage medium of claim 15, wherein the camera pixels are positioned such that a union of a FOV of each of the camera pixels included in the integrated display panel substantially covers a FOV of the integrated display panel.
 19. The computer readable storage medium of claim 15, wherein each of the camera pixels and the display pixels are mapped to a coordinate position, wherein image data is encoded with position data, wherein mapped pixels create a correspondence between camera pixels and display pixels with a similar FOV.
 20. The computer readable storage medium of claim 19, wherein the camera pixels and the display pixels operate simultaneously. 